Digital photofinishing system including scene balance and image sharpening digital image processing

ABSTRACT

A method of digital photofinishing comprising the steps of: producing a digital color image in optical printing densities of a color image captured on photographic media; processing the digital color image with a scene balance algorithm to produce a balanced digital color image; mapping the balanced digital color image through a hard copy media characteristic curve to produce a balanced digital color image mapped to print densities of the hard copy media; sharpening the mapped balanced digital color image with a sharpening algorithm optimized to avoid unacceptable artifacts; and digitally printing the sharpened digital color image in hard copy media.

FIELD OF THE INVENTION

This invention relates in general to digital photofinishing systems andmore particularly to a digital photofinishing system including scenebalance and sharpening digital image processing.

BACKGROUND OF THE INVENTION

Several problem areas need to be addressed when making color negativepaper prints from color negative film images. The object of the processis to make a pleasing print from the film image. The first problem is tofind the level of exposure necessary in a projection printer system toproduce that pleasing print. In the simplest implementation of aprocess, one projects the color negative film image onto aphotosensitive paper image receiver, processes the paper, and thenrepeats the process until a pleasing print has been obtained. Inautomated photo-finishing operations, a film scanner reads densitiesfrom the negative and passes the information to a computer algorithmthat computes the appropriate exposure values so that a pleasing printcan be obtained. At this point, the conventional photographic printingprocess ends. That is, if there are any remaining problems in aphotographic image, no other simple processes are available to reducethe severity of the problem. Problems such as poor scene balance, andpoor sharpness in the final image remain that we would like to corrector modify.

Methods and systems have been described for more than 10 years that aredevoted to producing pictorial images on various media and devices fromscenes captured on photographic film, via scanning to produce a digitalimage, image processing, and output rendering. Examples include thefollowing.

Journal of Imaging Technology, Vol. 14, Number 3, June 1988, Firth et.al. describe systems that capture scenes on film, scan film to produce adigital image, digitally process the image, and output via a laser AgXprinter.

U.S. Pat. No. 4,500,919, Schreiber discloses an image reproductionsystem that scans an image captured on film, displays the image on avideo monitor, enables image processing, and finally output to an inkedhardcopy.

U.S. Pat. No. 4,979,032 (Dec. 18, 1990, filing date: Dec. 27, 1988),Alessi et al. describe an apparatus, including a film scanner, an videomonitor, image processing, and output, to produce various outputvisually matched to the image displayed on the monitor.

U.S. Pat. No. 5,267,030, issued Nov. 30, 1993, inventors Giorgianni et.al. describe method and means to transform images captured on film, viadigitization on a film scanner, to a color metric or other space, withoutput onto a variety of media and devices. This document describes theimprovements offered by digital image processing, includingaesthetically pleasing modifications to the tone and color reproductionas well as sharpening.

U.S. Pat. No. 5,300,381, issued Apr. 5, 1994, inventors Buhr et. al.describe a pictorial imaging system that consists of capture onphotographic film, film scanning to produce a digital image, imageprocessing, and digital output.

U.S. Pat. No. 5,579,132, issued Nov. 26, 1996, inventor Takahashidescribes an image processing system devoted to storing or producingimages that have "substantially the same color" or additional "aestheticcolor correction" versus the original scene, based on a variety of imageprocessing transformations of the digitized image.

U.S. Pat. No. 5,608,542, issued Mar. 4, 1997, inventors Krahe et. al.describe a system that produces index prints based on scanning a filmframe, image processing, and rendering.

U.S. Pat. No. 4,945,406, issued Jul. 31, 1990, inventor Cok, describes asystem for achieving automatic color balancing of color images bytransferring color pixel values from log exposure RGB color values intoprinting density values and generating color correction offset valuesutilizing a printing density based color correction method.

The KODAK 35 mm/24 mm color negative film format Index Printer, sold byKodak, produces an index print (a matrix of small imagettes) reproducedfrom individual film image frames. The index print is produced by thephotofinisher when the original print order is processed and is suppliedto the customer as a convenient means of identifying image frames on thefilm (see: U.S. Pat. No. 5,608,542, above). The Kodak Index Printer usesimage processing on the miniature images including:

digital image in film RGB printing density

applying a scene balance algorithm to balance the digital film printingdensity image

mapping the color negative digital image onto a color paper (EDGE-type)characteristic curve

digital sharpening

rendering using a CRT printer onto photographic paper

In the Index Printer, the above image processing: (1) is not applied tofull frame images in a digital color printer; and (2) is not applied tohigh resolution images in a digital color printer, only low resolutionimages.

All of these articles or patents describe, in one form or another,processes for obtaining more pleasing prints from a film image capturethan the conventional optical process. There is thus a need for asolution to these problems, which can be incorporated into a digitalphotofinishing system.

SUMMARY OF THE INVENTION

According to the present invention, there is provided a solution to theproblems of the prior art.

According to a feature of the present invention, there is provided amethod of digital photofinishing comprising the steps of: producing adigital color image in optical printing densities of a color imagecaptured on photographic media; processing the digital color image witha scene balance algorithm to produce a balanced digital color image;mapping the balanced digital color image through a hard copy mediacharacteristic curve to produce a balanced digital color image mapped toprint densities of the hard copy media; sharpening the mapped balanceddigital color image with a sharpening algorithm optimized to avoidunacceptable artifacts; and digitally printing the sharpened digitalcolor image in hard copy media.

According to another feature of the present invention, there is provideda method of digital photofinishing comprising the steps of: producing adigital color negative image in optical printing densities of a colorimage captured on a color negative; processing said digital colornegative image with a scene balance algorithm to produce a balanceddigital color negative image mapping the balanced digital color negativeimage through a hard copy media characteristic curve to produce abalanced digital positive color image; sharpening the mapped balanceddigital color positive image with a sharpening algorithm optimized toavoid unacceptable artifacts; and digitally printing the sharpeneddigital color image onto hard copy media.

ADVANTAGEOUS EFFECT OF THE INVENTION

The invention has the following advantages.

1. A digital photofinishing system is provided that produces highquality digital photographic reflection prints from color negative filmimages at high print rates.

2. Sharpness that is difficult to correct in conventional opticalphotofinishing systems is corrected by digital image processing in adigital photofinisher.

3. Prints produced by the digital photofinishing system of the inventionwere preferred over prints produced by optical photofinishing systems.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a digital photofinishing systemincorporating the present invention.

FIG. 2 is a flow diagram of a preferred method of the present invention.

FIG. 3 is a graphical view useful in explaining aspects of the presentinvention.

DETAILED DESCRIPTION OF THE INVENTION

Referring to FIG. 1, there is shown a block diagram of a digitalphotofinishing system 10 incorporating the present invention. As shown,system 10 includes a digital scanner 12 for scanning color photographicmedia 14 to produce a digital color image of a color image frame 16 oncolor media 14. Color media 14 is a negative film. Image data manager 18processes the digital image to optimize the image for printing theprocessed image by printer 20 on color hard copy media 22. Color hardcopy media can be any high quality reflection or transmissive media,such as silver halide color paper, and media used in ink jet, thermal,electrophotographic printing processes. The operation of scanner 12 andprinter 20 are described in greater detail below.

Image data manager (IDM) 18 processes the digital color image fromscanner 12. IDM 18 is preferably a digital computer having user input 24(keyboard, mouse) and computer readable storage media input 26. Computerreadable storage media may comprise, for example, magnetic storagemedia, such as a magnetic floppy disc or magnetic tape; optical storagemedia, such as optical disc, optical tape or machine readable bar code;solid state electronic storage devices, such as read only memory (ROM)or random access memory (RAM); or any other physical device or mediumemployed to store a computer program. The digital image processingtechniques described below can be readable storage media. Alternatively,some or all of the techniques may be incorporated into programmable gatearrays or other hard electronic devices.

Scanner

The scanner 12 for the production of digitally processed prints scans a35 mm full frame image at a minimum resolution of m×n pixels. It wouldbe preferred that the scanner produce a higher image resolution of,e.g., 2m×2n pixels so that "panoramic" images or enlargements can beprinted with sufficient resolution without interpolation. The higherresolution is also preferred for the preparation of 5R prints. Allmagnifications higher than 5R will require higher resolution scans.

The scanner delivers the digitized data to the image processingalgorithm as "printing density." A scanner that measures printingdensity has red, green, and blue effective spectral response thatmatches that of photographic paper in an optical printer. Photographicfilms are designed based on the expectation that the capturedinformation in the film will be read by this type of red, green and bluespectral response characteristic.

Even though the scanner is not considered part of the digital imageprocessing path, some manipulations of the data may be required todeliver "printing density" to the processing algorithm. Two key stepsare converting scanner densities to calibrated scanner densities, andconverting the calibrated scanner densities to printing densities.Matrix operations may be required to perform these conversions, or alook-up-table can be used.

The first step in the process is to convert the raw scanner numbers tocalibrated scanner densities. If a matrix operation is used, anappropriate matrix correction model is shown below. ##EQU1## N_(rsx) israw scanner number and D_(sx) is corrected scanner density.

The next process in the scanner converts calibrated scanner densities tocalibrated printing densities. The process is implemented again with amatrix model that in this case can be up to a 3×10 matrixmultiplication. An example is shown below. ##EQU2## The output of thematrix multiplication is calibrated printing density. D_(sx) iscalibrated scanner density, and D_(px) is calibrated printing density.Either or both of these steps can be implemented with a 3Dlook-up-table.

According to a preferred method of the present invention as shown inFIG. 2, an image on photographic media (block 30) is scanned (by scanner12) to produce a digital image (block 32). After conversion to printingdensity, the digital image is sequentially processed by IDM 18 in animage processor section 33 with a scene balance algorithm (box 34) (See:Modern Exposure Determination for Customizing Photofinishing PrinterResponse," J. Appl. Photog. Eng., Vol. 5, No. 2, Spring 1979, pp.93-104.), print media characteristic curve mapping (box 36), imagesharpening (box 38) and then digitally printed on hard copy media (box4) by digital printer 20.

Hard copy media includes media optimized for the print technology used,i.e., laser or CRT photographic printers, ink jet printer, thermalprinter, electrophotographic printer, etc.

The scene balance algorithm is preferably performed as an analyticalsubsystem on a subsampled digital image (boxes 42 and 44) to produceshift parameters that are applied to the full resolution digital image(box 46).

Scene Balance Mapping

The process of operating a two stage film/media printing system has asits most imposing task that of finding a density value on the filmexposure and mapping the densities on that negative to the media so thatthe best overall density and color balance are obtained. The process isbest illustrated by tying together a series of reference points showingthe connections from a scene object to the photographic reproduction ofthe object.

FIG. 3, below, illustrates the concept of density tie points in ouranalysis of printing process. This process ties an initial target,usually a uniform gray card, to the photographic reproduction of thattarget. Test points are established for the initial exposure of thetarget object as an expected reference film density from a filmreference exposure for the test object. The quadrant labeled "Printer"merely reflects the film reference density to the log exposure axis ofthe media. The test target will be assigned a density to be achieved forthe reproduction of that object, or a density at the paper balancepoint. This point is characterized as the media reference exposure valueand a media reference density value.

The objective of the scene balance process is to first determine thefilm density value for the film reference exposure and to determine thedifference between this film density value and the media referenceexposure value. The difference, or Δ, is added to the film referencedensity. This process assures that the quadrant labeled "Print Through"will yield the appropriate print density in the final reproduction ofthe scene.

Real photographic opportunities, or images, usually do not contain testobjects that can be mapped in a formal process like that just described.Scene balance algorithms are designed to perform the task of estimatingthe reference film density, as if a gray card had been included in thescene. Once an estimate of the film reference density has been made, theprocess is duplicated just as described above. The examples show onlyone color record. Color films contain three records to capture the red,green, and blue information from a scene. Thus, the scene balancealgorithms must perform two tasks. First estimate the overall density ofthe film image to that the best neutral density is obtained in the finalprint, or print through quadrant of the scene. Then the balance betweenthe red, green, and blue exposure must be estimated.

The next step in the process is to map the balanced image through a hardcopy media (color photographic paper) characteristic curve. The printingdensity values of the balanced image are mapped to the appropriate printdensity values for negative photographic print paper. The finishedprocess produces an image that is color and density balanced, and in aprint density metric. The photographic paper curve is balanced.

Unsharp Masking Process

The final step prior to actually printing the image is the sharpeningprocess. An unsharp masking algorithm is applied to every image justprior to printing. The unsharp masking equation is as follows.

    D.sub.c (x)=D(x)+β(D(x)-∫Φ(ξ)D(x-ξ)dξ)

In this equation, D_(c) (x) is the "sharpened" density at position x,D(x) is the starting density at position x, the recommended value of βis 2.5, and the integral values at each pixel position are computed byconvolving the image with the kernel, below. ##EQU3## The red, green,and blue images are all sharpened to the same level.

The recommended value for β was established by adjusting the valueupwards until pictures, when printed, began to appear with unacceptablelevels of digital artifacts. These artifacts appeared as ringing, orhalos, on edges yielding images that appear unnatural.

This value of beta is then applied to a square wave target to measurethe modulation transfer function for the final processed image. The testto establish the full system MTF response is as follows. First, a targetis photographed using a color negative film. The original target has asquare wave pattern of approximately 40% modulation. The spatialfrequency response values for each frequency of square wave pattern is100% through all of the system visible frequencies. The film image wasscanned on the digital scanner to produce a 1024×1536 pixel digitalimage for subsequent processing. This image was processed through ourSBA plus sharpening path and printed. The spatial frequency response ofthe final print was measured with a high resolution microdensitometer,and the data analyzed using a harmonic analysis process. Table 1 liststhe red, green, and blue response measured in this test (average of foursamples) and represents the maximum MTF before significant artifactproduction occurs.

The MTF curve measured using this process represents the maximum spatialfrequency of any digital processing system for Digital Photofinishing.This as the final print image is a combination of camera lens, film,scanner, algorithm, print engine, and print media. Any combination ofthese elements that yields an MTF curve of this result, or any resultbelow the values listed in Table 1, will be considered as part of thisinvention. The set of parameters considered for this system, includingthe sharpening algorithm, are the maximum level of boost before goinginto a condition of oversharpening. Values of beta lower than thatspecified in the report are considered within the scope of thisinvention because these values also deliver prints that meet ourprinting requirements for making digital prints. If one of the systemcomponents is changed, then the beta value will be changed to assurethat any final image produced by this digital path will achieve theupper limit MTF curve.

                  TABLE 1                                                         ______________________________________                                        Maximum MTF Values                                                            Freq    Red            Green   Blue                                           ______________________________________                                        0.0     1.0000         1.0000  1.0000                                         0.5     0.9813         1.0500  1.0785                                         1.0     0.9680         1.1180  1.1423                                         1.5     0.8735         1.0840  1.0948                                         2.0     0.7118         0.924   0.9298                                         2.5     0.5263         0.7170  0.7170                                         3.0     0.3523         0.5380  0.5238                                         3.5     0.2225         0.3970  0.3725                                         4.0     0.1510         0.2910  0.2688                                         4.5     0.1150         0.2140  0.2023                                         5.0     0.0893         0.1610  0.1593                                         5.5     0.0690         0.1250  0.1308                                         6.0     0.0530         0.1000  0.1095                                         6.5     0.0410         0.0840  0.0945                                         ______________________________________                                         Notes:                                                                        (1) The column labeled "Freq" is spatial frequency in cycles/mm on a 4R       reflection print, and the columns labeled as colors are the response          values for that color at each spatial frequency.                              (2) The MTF values are given for a 4" × 6" print, 250 dots per inch     and 1024 × 1536 pixels.                                            

Psycho-physical experiments were conducted to compare these digital pathprints to a convention optical printing path. The optically printedimages were prepared using a CLAS 35 optical printer running in the fullorder printing mode, thus emulating the SBA operating in our digitalpath. The scene balance algorithm parameters used in the digital andoptical paths were the same so that similar prints could be prepared.Pairs of prints, digital and optical, were shown to a panel of threejudges who were asked to choose the best print from the pair. In 75% ofthe pairs, the digital print was selected, citing sharpness as thereason. The remaining 25% were the optical prints selected because ofgrain build-up in the digital prints.

Printer

At this point in the process, the printing densities should be fullybalanced and corrected. This image information can be printed through asimple printing density to print density look-up table.

In the simplest case (printing density to print density), the printingdensity values are mapped to the appropriate print density values for anegative photographic paper. The finished process produces an image thatis color and density balanced, and in a print density metric. We findthe most preferred mapping from the balanced digital color negativeimage onto the characteristic curve of the aim AgX paper results frommapping reference RGB printing densities representing an achromaticmiddle gray onto achromatic RGB paper densities. These RGB paperdensities are a function of the print material image dye spectra.

The final step prior to actually printing the image is the sharpeningprocess which has been described previously.

    D.sub.c (x)=D(x)+β(D.sub.x -∫Φ(ξ)D(x-ξ)dξ)

At this stage in the process, the encoded data is sent to a printingdevice that renders, or prints, the information. A calibration processmust be operating on this device such that the code values presented tothe printer will yield the expected print densities.

Printer Calibration

Printer calibration will be done as part of the image processing systemmaintenance such that test patch density differences between measuredand expected densities of less than 0.01 are obtained. The simplestembodiment only requires a neutral scale calibration. In more complexapplications, a color calibration may be necessary.

A series of uniform patches (at least 18) spanning the full range ofprinter code values are printed through an initial calibration LUT. Thisinitial LUT must cover all those D/A count values that produce densityon the print. The patch densities on the print are measured. With theinitial LUT, the list of code values, respective densities of thosepatches, and the aim curve, a new calibration LUT can be calculatedwhich should modify printing behavior according to the calibration aim.

It will be appreciated that image sharpening can be effected at otherpoints in the image processing chain. It will also be appreciated thatthe goal of the image processing described herein is to produce imagesvia digital printing that are similar to, or better than, imagesproduced by optically printing negative film images. This can be done ina straight forward manner as described herein when using digital hardcopy media having similar image dye spectra to the image dye spectra ofthe hard copy media (silver halide color paper) used in opticalprinting.

The invention has been described in detail with particular reference tocertain preferred embodiments thereof, but it will be understood thatvariations and modifications can be effected within the spirit and scopeof the invention.

    ______________________________________                                        PARTS LIST                                                                    ______________________________________                                        10         digital photofinishing system                                      12         digital                                                            14         color film                                                         16         color image frame                                                  18         image data manager                                                 20         printer                                                            22         color print paper                                                  24         input                                                              26         computer readable storage media input                              30         image on photographic media                                        32         produce digital image                                              34         process with scene balance algorithm                               36         print media characteristic curve mapping                           38         image sharpening                                                   40         digitally print image                                              42         create subsampled digital image                                    44         SBA density shift parameters                                       46         full resolution digital image                                      ______________________________________                                    

What is claimed is:
 1. A method of digital photofinishing comprising thesteps of:producing a digital color image in optical printing densitiesof a color image captured on photographic media; processing said digitalimage with a scene balance algorithm to produce a balanced digital colorimage; mapping said balanced digital color image through a hard copymedia characteristic curve to produce a balanced digital image mapped toprint densities of said hard copy media; sharpening said mapped balanceddigital color image with a sharpening algorithm optimized to avoidunacceptable artifacts; digitally printing said sharpened digital colorimage onto hard copy media; wherein said processing step includes thesteps of creating a subsampled digital image of said digital image,processing said subsampled digital image with said scene balancealgorithm to produce density shift parameters, and applying said densityshift parameters to said full digital image to produce said balanceddigital image.
 2. The method of claim 1 wherein said producing stepproduces a digital image in optical printing densities of an image framecaptured on transparent photographic negative film.
 3. The method ofclaim 1 wherein in said mapping step said hard copy media isphotographic paper and said hard copy media characteristic curve isphotographic paper characteristic curve, and wherein in said printingstep said hard copy media is photographic paper.
 4. A method of digitalphotofinishing comprising the steps of:producing a digital color imagein optical printing densities of a color image captured on photographicmedia; processing said digital image with a scene balance algorithm toproduce a balanced digital color image; mapping said balanced digitalcolor image through a hard copy media characteristic curve to produce abalanced digital image mapped to print densities of said hard copymedia; sharpening said mapped balanced digital color image with asharpening algorithm optimized to avoid unacceptable artifacts;digitally printing said sharpened digital color image onto hard copymedia; wherein said processing step includes the steps of creating asubsampled digital image of said digital image, processing saidsubsampled digital image with said scene balance algorithm to producedensity shift parameters, and applying said density shift parameters tosaid full digital image to produce said balanced digital image; whereinin said sharpening step said sharpening algorithm has a parameter whichis adjusted to a value near the value where unacceptable levels ofdigital artifacts begin to appear in the printed image.
 5. A method ofdigital photofinishing comprising the steps of:producing a digital colornegative image in optical printing densities of a color image capturedon a color negative; processing said digital color negative image with ascene balance algorithm to produce a balanced digital color negativeimage; mapping said balanced digital color negative image through a hardcopy media characteristic curve to produce a balanced digital positivecolor image; sharpening said mapped balanced digital color positiveimage with a sharpening algorithm optimized to avoid unacceptableartifacts; and digitally printing said sharpened digital color imageonto hard copy media; wherein said processing step includes the steps ofcreating a subsampled digital image of said digital image, processingsaid subsampled digital image with said scene balance algorithm toproduce density shift parameters, and applying said density shiftparameters to said full digital image to produce said balanced digitalimage.